Skip to content

Fix: safe downloads and export fallback timeout; prepare 0.1.3 - #6

Merged
nikolaos-paximadakis-eea merged 3 commits into
mainfrom
fix/safe-downloads
Nov 19, 2026
Merged

nikolaos-paximadakis-eea merged 3 commits into
mainfrom
fix/safe-downloads

Conversation

@nikolaos-paximadakis-eea

Copy link
Copy Markdown
Collaborator

Stacked on #5 (base harden/strictness); GitHub retargets it to main when #5 is merged and its branch deleted.

Fixes three findings from the Codex review of #5. These change behaviour (for the better), so they are kept out of #5.

  1. A failed download no longer destroys the existing file at dest. client.download() (sync and async) wrote straight to dest; a dropped connection or full disk left a partial file in place of the previous one. It now streams to a unique <dest>.<random>.part next to dest and replaces dest only when the body is complete (_core.atomic_destination). Present since the first release; affects every download-based method, including export_to_file.
  2. export_to_file fallback stays within one timeout. The v3 rerun on non-data-lake dataflows restarted the full timeout (timeout=300 could take ~600 s); it now gets only the time left.
  3. The fallback no longer uses a fixed <dest>.part, which overwrote a file of that name and collided with concurrent exports to the same destination.

Each fix has tests that fail before it (tests/test_safe_downloads.py, two new cases in tests/test_export_fallback.py); both fix commits pass on their own.

Also: version 0.1.3 and CHANGELOG (release after merge, as usual).

download() (sync and async) wrote straight to dest, so a dropped connection or
full disk left a partial file in place of the previous one. It now streams to a
unique '<dest>.<random>.part' next to dest and replaces dest only when the body
is complete (new _core.atomic_destination). Found in a Codex review; present
since the first release.
The v3 fallback restarted the full timeout (timeout=300 could take ~600 s) and
always wrote '<dest>.part', clobbering a file of that name and colliding with
concurrent exports. It now gets only the time left and writes through
atomic_destination. Found in a Codex review.
Base automatically changed from harden/strictness to main November 19, 2026 14:52
@nikolaos-paximadakis-eea
nikolaos-paximadakis-eea merged commit 62488ac into main Nov 19, 2026
6 checks passed
@nikolaos-paximadakis-eea
nikolaos-paximadakis-eea deleted the fix/safe-downloads branch November 19, 2026 14:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant